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ABSTRACT 

The final report on field testing of minicourses in under- 
graduate biology lists the objectives, procedures-^ applications 
and conclusions. Twelve schools mostly in the Midwest utilized 
minicourses developed at Purdue University in their regular 
classes* Data were collected and analyzed on student achievement, 
student attitude changes, and student reaction to the minicourse 
materials* The data were then used to revise and improve the 
minicourses where weaknesses in content or approach were detected 
during the field testing. A scale to measure student attitudes 

toward biology was developed by the project staff along with an 

/ 

instrument allowing students to record their feelings about 

i 

various aspects of a minicourse. The project demonstrated the 
positive consequences of using field-gathered data for improving 
the quality of instructional materials* 



INTRODUCTION 

In 1970 the Minicourse Development Project was instituted at 
Purdue University by Dr. S.N* Postlethwait. The project was 

^ spofas og^ by the National Science Foundation. During the first 

\j 

three years * efforts were concentrated on the design and develop* 
ment of minicourses . After student tryout of these minicourses 
at Purdue University* it was decided to launch a formal field 
testing program at twelve other colleges and universities* A 
two and one^half year evaluation grant was funded by the National 
Science Foundation in July of 1972* Actual testing of materials 
began in September of 1972* The field testing was coordinated 
by James Russell of Purdue's Department of Education and Don 
Tolliver of the Instructional Media Research Unit at Purdue under 
the leadership of Dr. Postlethwait. 
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STAT^MCNT OF THE PURrOSb: 

The general objectives of the evaluation phase of the 
Minicourse Development Project were: 

1* To provide corrective feedback information to the project 

staff about the effectiveness of the materials produced with 

suggestions for future development strategies* 
2. To determine whether the materials allow the students to 

master the objectives under various field conditions using 

a variety of student populations. 
3* To introduce the minicourse approach and disseminate the 

modular materials to several divergent types of colleges and 

universities . 

Thus, the purpose of the project was to determine the over- 
all effectiveness of minicourses rather than to compare mini- 
courses with conventional instruction* The basic questions to 
be answered were: "Are minicourses an effective form of instruc- 
tion?" and "Kow can their effectiveness be increased?" 

PROCEDURES 

Six minicourses were selected for use in the twelve partici- 
pating schools to provide a common base for evaluation* Each school 
selected approximately six additional minicourses to use during 
each term. The design for the field testing procedures is out- 
lined in Figure 1 (next page) • The evaluation phase of the 
project concerned itself with three general concerns: A) assess-- 
ment of the quality of the minicourses as perceived by students 
using tliem, B) assessment of student achievement as a function 
of using the minicourses, and C) assessment of changes in student 
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Figure 1* Experimental design for field testing of Audio-Tutorial 
Minicourses* 
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abbitiulos Unvard biolo<jy as a roiiult OL the uuiiic<.HiL\50 moi.lo ot 
instruction* 

A) Assessment of minicourse quality 

The evaluation attempted to determine whether or not students 
felt: 

(1) The objectives of minicourses were achieved; 

(2) the audio portions, Study Guides, handouts, visuals, 
specimens and models were of high quality; and 

(3) The minicourses were relevant to student needs* Such 
information was useful to developers in making revisions 

; and modifications in minicourse content and activities* 

/ The main instrument used to assess minicourse quality is 
cdntained in Appendix A* Itie following five-point scale was 
used to record responses* 

SA = Strongly Agree - 5 
A = Agree = 4 

N = Neutral - 3 

D = Disagree = 2 

SD = Strongly Disagree " 1 

For each of the questions, the frequency of response (A, B, 
C, D, E) was determined* Also, the mean was computed for each 
response* A sample computerized printout of this data is contained 
in Appendix B , 

This questionnaire, designed to assess perceptions of minicourse 
quality, was administered to a variety of populations* Such 
populations varied in terms of total enrollment in the respective 
colleges or universities, class size, and student academic 
background and abilities* 



The evaluation ph aso of the project was also concerned with 
determining to what extent students were able to master minicourse 
content. Achievement measxjres were available for most of the 
minicourses tested. In order to assess achievement^ a pretest^ 
posttest experimental design was used. Generally^ students were 
given a pretest at the start of a term^ were given a posttest 
immediately after completing each minicourse^ and were administered 
a delayed posttest at the end of the term. Thus, it was possible 
to determine how much the students had learned from the minicoursfe 
by, comparing gains in achievement from the pretest to the immediate 
posttest. It was also possible to determine the amount of 
retention by comparing immediate posttest scores with delayed 

i 

posttest scores* A sample of achievement results for one semester 
at one of the participating schools is shown in Appendix C. 

A necessary part of this phase of the evaluation concerned 
the development of valid instruments to assess achievement. 
Therefore^ for each achievement test/ it was necessary to examine 
item difficulty^ effectiveness of distractors/ and the ability of 
the items to discriminate between high^scoring and low-scoring 
students, in shorty it was necessary to conduct an extensive 
item analysis. A sample of such an item analysis of one mini- 
course achievement test is shown in Appendix D. 
C) Assessment of student attitudes 

A third aspect of the evaluation project concerned the assess" 
ment of student attitudes toward biology as a function of the 
minicourse mode of instruction. Specifically ^ the purpose of this 
aspect of the evaluation program was to determine the extent to 
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wtiicli the minicouvuos influenced sLiuiont ^KtiLudo:^. Althougti the 
assessment of attitudes was a less integral part of the evaluation 
program than the assessment of minicourse quality or student 
achievement, it was thought that such information would be 
valuable to developers and future users of the minicourses* 
Appendix E contains the student attitude scale which was developed 
by the Minicourse Development Project evaluation staff* 

APPLICATION AND DISSEMINATION 

The primary application of the findings from the field testing 
was in the revision and improvement of the minicourses before 
their release for publication* The data from the student tryout 
wore forwarded to BSCS in Boulder, Colorado, where a team of 
instructional developers under the direction of Dr. Postlethwait 
implemented the necessary changes within the minicourses* Where 
major revisions were necessary, the minicourse was re-evaluated* 

The field testing and evaluation procedures developed as a 
part of this project were applied to a number of additional 
projects in a variety of instructional disciplines. Dr * Russell 
used the procedures with minor modification as part of a U*S. 
Public Health Service grant to develop minicourses in medical^ 
surgical nursing* The techniques were also used by the Laboratory 
for Applications of Remote Sensing at Purdue University* In 
addition, Dr* Tolliver applied the appropriate parts of the 
research in an EXXON Education Project which he directed* The 
EXXON Project developed twelve minicourses to be used for pre- 
service and in-service teacher training* 

The procedures, techniques and results of the evaluation of 
minicourses in undergraduate biology have been presented at six 



national c<.>n\\Mitioni5 anj in nuraoi\Hi^ pub L (ace Api.ici:JiK r) , 

In addition to the minicourses which will be published and 
distributed/ two additional developments being used widely are 
instruments developed by the evaluation staff* They are the 
Minicourse Evaluation Form (Appendix A) and the Biology Attitude 
Scale (Appendix C) » Numerous requests have been received for 
their use in other research since no similar instruments are 
currently available* Both were included as examples in Modular 
Instruction (Burgess Publishing Company^ 1974) * Modifications of 
both forms have been used in numerous research and evaluation 
studies * 

CONCLUSIONS 

As part of the evaluation phase of the Minicourse Development 
Project/ twelve schools evaluated different minicourses developed 
by the Project* The intent was not to evaluate every minicourse 
produced by the Project^ but to evaluate only a sample because of 
limited available funding* As a consequence of the field testing 
of the minicourses^ the minicourses were revised based upon the 
feedback received* Consequently ^ the Project has demonstrated 
that it is feasible to have "experts" develop and evaluate 
minicourses so that students in schools not able to produce such 
materials can benefit from them* The development teams included 
a subject matter specialist^ an educator and students* The 
evaluation team included an educator^ a researcher and a 
psychologist* Based upon the results of this study ^ it appears 
this is a feasible approach to evaluation* 

The evaluation procedures also resulted in the dissemination 
and utilization of the minicourses developed by the Project* Ten 
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of the iMVticipatin^j sclux^ls have ^unce adopted tUo Kiiuicoutso 
approach in teaching part or all of their introductory biology 
courses* One school which had taught all of their undergraduate 
biology in a traditional lecture-lab approach is now teaching 
three of their courses in a totally minicourse approach* 

Two direct contributions of the evaluation procedures were 
the development of two necessary evaluation instruments — the 
Minicourse Evaluation Form and the Biology Attitude Scale* These 
instruments were designed and validated by the evaluation staff* Since 
then they have been widely used in other research and evaluation 
studies * 

SUMMARY 

This report provides a brief introduction to the evaluation of 
minicourses in undergraduate biology* It includes a statement of 
purpose, outlines the procedures, discusses applications of the 
research, and provides a concluding statement* The overall purpose 
of the research was to systematically evaluate a series of 
minicourses developed by the Minicourse Development Project at 
Purdue University* 

Each of the twelve participating schools evaluated approximately 
a dozen minicourses each term* The achievement of the students 
was measured along with any changes in their attitudes toward 
biology* The studentri' reactions to the content and components 
of each minicourse were reported* 

.The evaluation data were used by a team of instructional 
developers headed by Dr* Postlethwait to revise and modify the 
minicourses prior to publication* Other contributions of the project 
include development and testing of a field-based evaluation procedure 



which lu\s j^inco boon applio^l to otUor inutiuotional Uovclopmont 
projects. In order to carry out the necessary evaluative 
procedures * the project staff developed and validated two 
instruments — the Minicourse Evaluation Form and the Biology 
Attitude Scale. 

The feasibility of an extensive field-based evaluation program 
has been demonstrated. Dissemination and adoption procedures 
have also been tested. Problems were uncovered and solved in 
most cases, The minicourse approach has been used successfully 
in a variety of instructional situations and with very diverse 
student populations. 
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APPENDIX A * 
MINIC0URS6 EVACUATION 



RCV.U7; 



Miiikntiru* dcvclopiiiL'nl tv j Inn^ uni) ottcn putt^uktiit: i^mhis^ The mintci^ursc Ihjl yuu huvC }u5t Completed has undetp^he 
Mvcnl niodiliLitionv jnd rcrmcutaih. prtnurtty ditcvtvd tlic LunstruCttve critict^m ol^ students in tUc past. As un ititcgrat p«t of 
ouf iiru^Nin of minunur^e dcvcliipment, YOUR lielp needed. 

Please rcud cjlIi item CareTiilfy uiiJ choo^W the unsw^r (A. I), f . I), or F) i\u* iiviu^s itttu ft to yottr fcchngson tlit: matter, Mch 
ydu liuve made your dniue. durki^n iit the jppnipiutc hruikiM Tor xUc voncvpoiuSttii; itiin ON Ull ('OMPCTUR CARD. DO HOT 
writr tvti the itucsttoiiruitc it^etf AUti. sime wo are only iiitL^resti J m your I'KANK ivjliutiint (vt tlti^ miniLiiurse. plL*a>c DO NOT 
wnte yuuT name on the computer curd but please DD be honest in your evjtu.itioii IndtcatL* the title ol thi* miniLOursc oti the TOf of 
the computer card. 

Finally, a scpjrjte COMMENT SHEET ts provided for any additional Comments, criticisms und suggestions you may Ciie to 
make with respect to the minicourse. Your cooperation is appreciated, 

Delow are 2 example items The fifst item contains a standardized rating scale with 5 unchor points: Strongly Agree (5A): 
Agree (A). Neither A^fce, Nor Disagree (N): Disagree (D), Strongty Disagree iSD). 

bxample A 

SA A N 0 SO. 

The United States is in need of a Nat»ona! Health Program. (If yo** neither ag*ec. nor disu^ree. 

bidcken m column C on the IBM card,) A 8 f D E 

The second type of item on thl^ questioniuire contuins jn open ended stjtem^nt to which you wilt be rei^uired to respond more 
specific ally. 

Example 6. 

Current progress in developing a N;itional Health Pfogrum has been 



A 

Too slow 



6 



C 

About rigfit 



E 

T<io Fatf 



(I r your fcehngs more closely Correspond to jbout right, blucken m column C on ihe IBM cjrd.) 
OBJECTIVES 





SA 


A 


N 


0 


SO 


1 , The objcctivci were very Cleaily iiated. 


A 


B 


C 


D 


E 


2. The maieriiili preiented were retried to the objeciivei. 


A 


B 


C 


D 


E 


3. 1 reel thai (he ilated objeetivei Tor thii minicoufie were achieved. 


A 


B 


C 


D 


E 


AUDIO 














SA 


A 


N 


0 


SO 


4. Voice quality wai very clear. 


A 


8 


C 


D 


E 


5. Speiikui'i mood wai very enlhuiiaitic. 


A 


8 


C 


D 


E 


6 lii^iruciioni were eteai <in(f eaiy to follow. 


A 


8 


c 


D 


E 


7. In^iruetioni for the exeiciiei/aclivitiei were adequately expUined Tor my knowledge of (he 












topic. 


A 


6 


c 


D 


E 


8. Pacing 












A B C D 










E 



Too slow 

9, Overall evaloiition of audio preu*iit3tion 



Ahout right 



Too fast 



A 

Excellent 



B 

Good 
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c 

Averafif 



Below average 



E 
Poor 



1 1 



srUDVwmi- andhanixiuts 





SA 


A 


N 


0 


SO 


10, The wfiUcn maurul was clearly presented. 


A 


B 


c 


D 


E 


1 L The written matendl was brief and to the point. 


A 


B 


c 


D 


E 


12, The study guide and 4udio tape were very well synchronized. 


A 


B 


c 


D 


E 


VISUALS (photos, charts and diagr;ims) 












Niite, Ifttiere were no visuals in this minkourse. leave questions 13-15 Mank. 














SA 


A 


N 


D 


SO 


13, tti general, the visuals were of excellent qu;ility <clirtty} 


A 


B 


c 


D 


E 


14. The visuals were smuuthly presented ;iiid integrjted within the sequence uf the mmicourse. 


A 


B 


c 


D 


E 


iS. The visuals were ciJccmt m ci>ntributnig tu my underst;inditig uf rhe subject matter. 


A 


B 


c 


D 


E 


TANGIBLES (Specimens und models) 












Nute^ If tfitie were nu t^ntcibles 111 tins mmicrmrtf lc;ive ouestiuns 16*17 blink. 














SA 


A 


N 


D 


SO 


\h In gencTiil. the ungibles were smuutltly pTeseuteil jiiid iiiTCgnTcd wiihin ttie sequence of the 












ntiiiicuurse. 


A 


B 


c 


D 


E 


17, The ungibles were effcciivt m euiitTibutiiijj lu my uiidersuiidiii|t of die subjeet fn;itier. 


A 


B 


c 


D 


E 


GENER/^L 


SA 










A 


N 


0 


SO 


18. t fpund this miiitcuursc very siimuUtiiig. 


A 


B 


c 


D 


E 


19. This mimcourse w«very urgimzed. 


' A 


B 


c 


D 


E 


ZO- Titis mtiiieuurse was relevant to my interests. 


A 


B 


c 


D 


E 



21. 1 spent dppruxim^tely im thisitiimcoursv* 



A B C D E 

Les^ :han I ttour 1^ liuurs 2 huurs more than 

tiour 2 Itoun 

Please inAe ^uy aUUitiondl coititnctits. cnttcisms and suggcsttot^^i relevant to this m»uCoiifSC on the COMMENT SHEET. 

Tlianks ;ii^in lor your coopcr^rn^i' 
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SAMPLE DATA FOU ACltl IIVLIMLNT UtltJULTS 

BIOLOGY 



MINXCOURSE 


Pre 


Post 1 


Post 2 


Complementarity of 
Structure and Function 
(6)** 


Leaf (8) 


18 


71 


72 


Meiosis (8) 


9 


57 


48 


Multicellularity (7) 


26 


46 


74 


Transformation of 
Energy I (8) 


33 


67 


71 




Means 


22 


60 


66 



♦Scores given in percentages (rounded off to nearest percent) 
♦^Nuinber in parenthesis denotes ntuiiber of items 
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SAMPLE 


ITEM ANALYTIC 


DATA FOR ACHIEVliMliNT 


TliSTS 


ITEM 


PROPORTION CHOOSING EACH 


ALTERNATIVE 




A 


B 


C 


D 


E 


1 
2 
3 
4 


0.174 
0.391* 
0.130 
0.348* 


0.0 
0 .043 
0.304 
0.304 


0.783* 
0.478 
0.435* 
0.217 


0.043 
0.087 
0.130 
0.130 


0.0 
0.0 
0.0 
0.0 


5 

6 

i 


0.174 
0.0 


0.348 
0.0 


0.391* 
0.043 


0.087 
0.957* 


0.0 
0.0 


ITEM 




ALTERNATIVE CORRELATION 


WITH TOTAL 


SCORE 


J 

i 


A 


B 


C 


D 


E 


I 
2 
3 
4 


-0.600 
0.559* 

-0.507 
0.509* 


0.0 
-0.493 
-0.107 

0.082 


0 .478* 
-0.292 

0.436* 
-0.267 


0.149 
-0.094 

0.011 
-0.507 


0.0 
0.0 
0.0 
0.0 


5 
6 


-0.140 
0.0 


0.326 
0.0 


0.113* 
-0.065 


-0.558 
0.065* 


0.0 
0.0 
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Please rate each statement on the extent to which you agree. For 
each, you may: (A) Strongly Agree, (B) Agree, (c) be Undecided, 
(D) Disagree, or (E) Strongly Disagree. 

After you have made your choice, blacken in the appropriate 
response in the columns on the IBM card corresponding to each item. 

A B C D E 

Strongly Agree Undecided Disagree Strongly 

Agree Disagree 

1. Biology is very interesting to me, 

2. I don ^ t like biology, and it scares me to have to take it. 
p. I am always under a terrible strain in a biology class. 

r 

* j4. Biology is fascinating and fun, 

5. Biology makes me feel secure, and at the same time it is stimulating 

6. Biology makes me feel uncomfortable, restless, irritable and 
impatient. 

7. In general, I have a good feeling toward biology. 

8. When I hear the word biology, I have a feeling of dislike. 

9. I approach biology with a feeling of hesitation. 

10. I really like biology, 

11. I have always enjoyed studying biology in school. 

12. It makes me nervous to even think about doing a biology experiment. 

13. I feel at ease in biology and like it very much, 

14. I feel a definite positive reaction to biology; it's enjoyable. 
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Below are £>oinc soulc:? on which wo 'v.\^i*j.a ixA^' ; to rati" 
your feelings toward biology. On each scale, you c?n rate your 
feelings toward biology as a 1, 2, 5, 4, or 5, THERE ARE KO 
CORRECT ANSWERS. Also, some of the scales soe:n to iralce more rcnrc 
than others. Don*t worry about it. Just rate youj feelings tovmrd 
biology on these scales as best you can. Pierce don't leave any scalr ^ 
blank. 

For your response to each scale, blaclten in the appropriate 
box on the XWL card. 
BIOLOGY IS: 



15. 


Good 


A 


B 


C 


D 


E 




16. 


Clean 


A 


B 


C 


D 




Dir'or 


17. 


Worthless 


A 


B 


C 


I> 


E 


Valur.'ole 


18. 


Cruel 


A 


B 


C 


D 


* ^ 


Kind 


19. 


Pleasant 


A 


B 


C 


D 


U 


U,'!pl.'''sant 


20. 


Sad 


A 


B 


C 


D 


E . 




21. 


Nice 


A 


B 


C 


D 


S 




22. 


Fair 


A 


B 


C 


D 


E 


Unfair 
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APPENDIX F 

Presentations at National Conventions : 

"The Effect of Pretesting on the Achievement of Students using 
Audio-Tutorial Minicourses*' by James Russell at the 1974 National 
Convention of the Association for Educational Communications and 
Technology, Atlantic City, Nw Jersey^ March 19^ 1974 

"Minicourse Development Project; First Year of Field Testing in 
Review" by James Russell at the Fifth Annual International 
Audio-Tutorial Congress Conference at Ohio State University^ 
Columbus^ Ohio^ November 6^ 1973 

"Evaluation Techniques in Audio-Tutorial Instruction" by James 
Russell at the Association for Educational Communications and 
Technology National Convention^ Las Vegas ^ Nevada^ April 9^ 1973 

"The Development of a Minicourse Evaluation Form", by James Russell 
at the Fourth Annual Convention of the International Audio-Tutorial 
Congress^ Columbia^ South Carolina^ November 3^ 1972 

/ 

Multi-media presentation on the Audio-Tutorial System for the 
Biological Sciences including evaluation by James Russell at the 
Tenth Annual Convention of the National Society for Prograinmed 
Inistruction at New Orleans^ Louisiana^ March 16^ 1972 

"The Evaluation of Minicourses during Field Testing" by James Russell 
at the Third Annual Audio-Tutorial Systems Conference at Purdue 
University^ Lafayette, Indiana^ November 2, 1971 



Publications and publications pending ; 

"The Development^ Validation and Application of Attitude Scales'* 
by James Russell and Stephen Hollander submitted to Improving 
Human Performance : A Research Quarterly of the National Society 
for Performance and* ^ lnstruction 

"Field Test Results of Six Minicourses Evaluated at Olivet Nazarene 
College During the Spring of 1973" by Harry Fulton and James Russell 
submitted to The American Biolog y Teacher 

"Techniques for Evaluating Minicourses" by James Russell submitted 
to The American Biology Teacher 

"The Effect of Pretesting on the Achievement of Students using 
Audio-Tutorial Minicourses" by James Russell submitted to Th e 
American Biology Teacher 



